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ABSTRACT 

A survey was conducted of Colorado secondary career 
and technical programs in 1995 to gauge how the state has responded 
to mandates of the Carl Perkins Vocational and Applied Technology Act 
of 1990. Results were reported for the first three of nine standards 
developed by a statewide committee in 1992. The three standards 
require secondary programs to demonstrate student achievement in 
career and technical programs in the following areas: (1) basic and 
applied academic skills; (2) general occupational skills; and (3) 
specific occupational skills. Information was gathered through a 
survey questionnaire mailed to each secondary provider with a 
foilow”up, resulting in returns from 92 percent. In the area of 
standard one, the survey results showed that programs use both 
commercial and locally developed instruments to measure attainment in 
this area. About half of the programs had results available, with 
about 65 percent of those gathered from exit data of program 
completers. For standard two, programs use a range of methods to 
assess general occupational skills, with most programs using more 
than one method of assessment. The most frequently occ^:T“ring method 
is tests, followed by checklists, "other," portfolios, and skills 
profiles. Forty-two percent of secondary programs indicate that they 
use pre- and posttests to assess general occupational skills whereas 
the remainder use other methods. About 54 percent of the programs 
were able to document student general occupational skills attainment. 
For standard three, programs use a variety of methods to assess 
specific occupational skills. The most frequent method is 
locally-developed assessments, followed by successful completion of 
the program, certification examinations, commercial instruments, 
portfolios, and "other." Nearly 38 percent of secondary programs 
indicate that they use pre- and posttests to asses specific 
occupational skills whereas the remainder use other methods. About 56 
percent of the programs had results available to document student 
specific occupational skills attainment. (KC) 
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RESULTS, SECONDARY SURVEY OF VOCATIONAL 
STANDARDS ONE THROUGH THREE 

February 12, 1996 

This reports the results of the survey conducted in 1995 of all CCCOES 
approved secondary career and technical programs in Colorado. The 
intent of the survey was to gauge how Colorado has responded to 
mandates under the Carl Perkins Vocational and Applied Technology Act 
of 1990. These requirements made it necessary to create and carry out a 
statewide system of standards and measures. This report deals with the 
first three of nine standards developed by a statewide committee in 1992. 

Designed to allow flexibility at the local level, Colorado Standards One 
through Three require secondary programs to demonstrate student 
achievement in career and technical programs. They include 1) basic and 
applied academic skills, 2) general occupational skills, and 3) specific 
occupational skills. No statewide profile has been available prior to the 
results reported here. 

The survey questionnaire was mailed to each secondaty provider in May 
1995. A follow-up mailing to non-respondents was mailed in August 
1995. CCCOES staff was involved in personally contacting non- 
respondents. By November 1 995, 934 programs had returned 
questionnaires, representing 91 .9% of all secondary career and technical 
programs in Colorado. 

The pages immediately following are intended to provide highlights frorri 
this survey. The last section of this report contains more survey detail in 
spreadsheet form. 
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How have institutions responded to Standard One ? 

Standard One deals with demonstrating student gains in basic and applied 
academic skills. This standard deals with how well programs integrate basic 
skills within the career and technical 



curriculum. 

Programs use both commercial (72%) 
and locally-developed (16%) 
instruments or assessments to 
measure student attainment in this 
area. About 11% of programs report 
they use "other" sources of 
measurement to determine gains in 
basic and applied academic skills, a 
finding that is puzzling given that these 
prograrhs did not specify alternative 
methods. 



Sources of Measurement 
Basic and Applied Academic Skills 
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Samples v. All Students 

Basic and Applied Academic Skiils 



Programs also were asked whether they 
measured basic and applied academic 
skiils of all students or samples of 
students. Responses suggest that 
secondary programs are more likely to 
assess all students (69%) than their 
postsecondary counterparts (51%), 



Sarroles (31.38%)- 



^Al Students (68 62%) 



According to this survey, under half 
of all secondary programs had 
results, either gain scores or other 
data, to report for Standard One. 

The basic skills area most frequently 
mentioned was "other," followed by 
reading, math, and English. It is not 
certain what programs mean when 
they indicate they have available 
results in "other" than traditional 
basic skills areas. 



Availability of Results 
Basic and Applied Academic Skills 
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There are several techniques for 
demonstrating gain in basic and 
applied academic skills. 
Techniques for demonstrating gain 
utilize exit testing. Sixty-five (65) 
percent of secondary career and 
technical programs indicate they 
use exit testing. The survey was 
unable to determine what 
techniques programs not using 
exit testing utilize to demonstrate 
gain. 



Exit Testing 

Basic and Applied Academic Skills 



No Exit Testing (35.m)- 



“ Ex* Testing (64 62%) 



How have institutions responded to Standard Two ? 

Standard Two deals with demonstrating student attainment in general 
occupational skills. General occupational skills are those skills needed for 
success in any workplace, no matter what career is chosen. 



Programs use a range 
of methods to assess 
general occupational 
skills. Most programs 
use more than one 


Other 


method of assessment. 
The most frequently 


Portfolios 


occurring method is 


Tests 


tests, followed by 
checklists, "other," 


Skills profiles 


portfolios, and skills 


Checklists 


profiles. The category 
"other" may include 
demonstrations or 
other forms of rating. 





Sources of Measurement 
General Occupational Skills 
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Forty-two percent of secondary 
programs indicate that they use 
pre- and post-tests to assess , 

genera! occupational skills whih 
the remainder sse other methods. 

Unlike basic and applied 
academic skills (Standard One), 
this standard does not require that 
students demonstrate a gain. 
Nonetheless, many secondary 
programs suggest those gain 
scores for general occupational 
skills are in use. 



Pre- and Post-testing 
General Occupational Skills 
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Availability of Results 
General Occupational Skills 



Programs were asked to specify 
whether results were available to 
document student general 
occupational skills attainment. Almost 
54 percent of the programs indicate 
availability of results. Such results are 
required for program approvals granted 
after July 1, 1995. 



Unavaaablo (46.3S%) 



How have institutions responded to Standard Three ? 




AvaSable (53.«2%) 



Programs use a variety of methods to 
assess specific occupational skills. 

By far, the most frequently occurring 
method is locally developed 
assessments, followed by successful 
completion (of the program), 
certification examinations, commercial 
instruments, portfolios, and "other." 



Sources of Measurement 
Specific Occupational Skills 
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Nearly 38 percent of secondary 
programs indicate that they use 
pre- and post-tests to assess 
specific occupational skills while 
the remainder use other methods. 
Unlike basic and applied 
academic skills (Standard One), 
neither general or specific 
occupational skills standards 
require that students show a gain. 
Nonetheless, many secondary 
programs suggest that they 
produce gain scores for general 
and specific occupational skills. 



Pre- and Post-testing 
Specific Occupational Skills 



-Pre & Post (37.46%) 



Other (02.54%)- 



Availability of Results 
Specific Occupational Skills 



Programs were asked to specify 
whether results were available to , u n \ o.\)( ‘ 
document student specific ' . 

occupational skills attainment. 

Almost 56 percent of the programs 
responding indicated availability of 
results. Such results in this area 
are required for program 
approvals granted after July 1 , 

1995. 
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Sources of Measurement for Basic and Applied Academic Skills 






American College Testing (ACT)A/l tjfK Keys 
California Achievement Test (CAT) 
Comprehensive/Califomia Test of Basic Skills (CTBS) 
High School Completion Test (HSCT) 

Iowa Test of Basic Skills (ITBS) 

Pre-/Post-CORD Materials 

Stanford Achievement Tests 

Test of Achievement and Proficiency (TAP) 

Test of Adult Basic Education (TABE) 
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Checkiist of basic and applied academic skills 

Test of basic and applied academic skills 

Student portfolio demonstrating basic skills proficiency 






All students at program entry 
All students at program exit 
A sample of students at program entry 
A sample of students at program exit 



Ail students at program entry and exit 
Samples of students at program entry and exit 



Achieving English scores of [n] or better 
Achieving mathematics scores of [n] or better 
Achieving reading scores of [n] or better 
Achieving [other academic area] scores of [n] or better 
Other results for program completers 



Achieving English scores of [n] or better 
Achieving mathematics scores of [n] or better 
Achieving reading scores of [n] or better 
Achieving [other academic area] scores of [n] or better 
Other results for program enrollees 
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Colorado Secondary Vocational Standards, Survey Results 

Standard Two-^Generat Occupational Skills 

Sources of Measurement for General Occupational Skills 

% of total n 






Measurement Strategies for General Occupational Skills 

% of total n 



23% 216 All students at program eritp/ 

45% 428 All students at program exit 

1 4% 137 A sample of students at program entry 

35% 332 ■ 

24% 227 All students at program entry and exit 

1 8% 1 68 Samples of students at program entry and exit 



Average Results for General Occupational Skills 



83.6 % achieving [75.1]% of general occuoational skills 

83.7 % achieving [76.0]% of general occupational skills 
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9% 87 Standardized 
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46% 442 Performance 
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Standard Three-Specific Occupational Skills 

Sources of Measurement for Specific Occupational Skills 



% of total 


n 


61% 


584 


4% 


37 


41% 


387 


50% 


478 


4% 


42 


2% 


19 


34% 


325 


54% 


519 


6% 


61 


Vleasurement 


% of total 


n 


21% 


201 


50% 


473 


13% 


122 


35% 


330 


22% 


214 


15% 


143 


8% 


77 



Locally-developed 
Standardized 






Written 

Performance 



Written 
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All students at program entry 
All students at program exit 
A sample of students at program entry 
A sample of students at program exit 






All students at program entry and exit 
Samples of students at program entry and exit 
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Average Results for Specific Occupational Skills 



Avg. 



84.4 % achieving [75.8]% of general occupational skills 

% ach ievin g [92.5] score on cert ification/iicensure examinations 

■ l■llll■i I 






82.8 % achieving [77.2]% of general occupational skills 



NOTE: Percentages may not sum to 100% because of multiple survey responses 
srvstnd2.wb1 /secsum 
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